REMARKS 

Claims 91-130 were pending. Claims 131-152 stand withdrawn. 
Claims 91-130 stand rejected. Claims 92, 100, 107-120, and 126-152 were 
cancelled. Claims 91, 93, 101, 106, and 121 were amended. Claims 153-172 were 
added. Claims 153-172 remain in the application. 

Claims 92, 100, 107-152 were cancelled. Claims 93 and 101 were 
amended to incorporate language formerly in Claims 92 and 101, respectively. 

In the Decision on Appeal, new grounds of rejection of Claims 91- 
105 and 121-130 were made under 37 CFR § 41.50(b). The Board stated: 

"We make the following new grounds of rejection using our 
authority under 37 CFR § 41.50(b). 

"Claims 91-105 and 121-130 are rejected under 35 U.S.C. 
1 12, second paragraph, as being incomplete for omitting essential 
elements, such omission amounting to a gap between the necessary 
structural connections. A claim which fails to interrelate essential elements 
of the invention as defined by applicant(s) in the specification may be 
rejected under 35 U.S.C. 1 12, second paragraph, for failure to point out and 
distinctly claim the invention. See In re Venezia, 530 F.2d 956, 957-58, 
189 USPQ 149 (CCPA 1976); In re Collier, 397 F.2d 1003, 1105, 158 
USPQ 266, 267 (CCPA 1968). 

"All of Appell^ts method claims share the same problem in 
that they are not limited to a machine implemented method. Normally no 
such limitation is required. However, in the situation before us. Appellants 
have specifically stated that the invention "provides the fundamental 
advantage of eliminating the need for manual intervention." See 
Appellants' specification at lines 1-2 of page 6. Appellants* method claims 
are not currently so limited. 

"We do not read the claim language "automatically" as 
requiring a machine. For example, the "automatically identifying" in line 3 
of claim 91 does not require a machine as it also covers a manual operation 
performed in a rote or mechanical fashion by a human. We also note that 
the mere complexity of the steps in Appellants' method claims does not in 
itself require that a machine perform them." 

The independent method claims overcome this rejection as follows. 



Claims 91 and 121 were amended by adding: 

"wherein said steps are without manual intervention and 
said altering follows any and all identifying of said main subject." 
This language is supported by the application as filed, notably at page 21, lines 1-5 
and page 5, line 27 to page 6, line 2. Added Claim 157 similarly states: 

"wherein said identifying and altering steps are without 
manual intervention. " 
and added Claim 164 states: 

"wherein said steps are without manual intervention." 
In the final rejection sustained by the Board, Claims 91-130 stood 
rejected under 35 U.S.C. 102(e) as being anticipated by U.S. Patent No. 6,504,951 
to Luo et al. The final rejection stated: 

"As to claim 91, Luo discloses a method for modifying an 
image having pixels, comprising: 

"automatically identifying a main subject (candidate sky) of 
the image (figures 6, 73-74, column 9, lines 25-35); and 

"automatically altering pixel values (eliminating pixel 
having tecture above threshold or fills voids of connected components) to 
emphasized the main subject, said altering following identifying (abstract, 
figures 6, 75, columns 9, lines 38-39, 57-67 note that thresholding, 
eliminating, and region growing are conducted after identifying the main 
subject, candidate sky). 

"As to claim 92, Luo further discloses the step of identifying 
comprises segmenting the image into a plurality of regions (col. 9, lines 25- 
43). 

"As to claim 93, Luo further discloses generating a belief 
map of said region (fig. 6 71, col. 9, lines 15-27, col. 8 lines 60-64). 

"As to claims 98-99, Luo further discloses altering 
emphasizes the main subject are part of said main subject (abstract, fig. 6, 
75, col. 9, lines 38-39, 57-67, note that filling voids of connected 
components that is main subject, and eliminating connected components 
that a not blue sky). 

"As to claims 100-101, Luo further discloses desaturating 
the pixels that are not a part of said main subject (col. 16 lines 3-14) and in 

-10- 



desaturating further comprises: calculating luminance values of the pixels 
that are not a part of main sibject and replajdng the color values of the 
pixels that are not a part of the main subject with respective luminance 
value (abstract, fig. 6, 75, col. 9 lines 38-39, 57-67, col. 15 hne 65-col. 16 
line 14). 

"As to claims 102-104, Luo further discloses enhance 
saturation of the main object pixels (sky) (and inverting the hue values of 
pixels corresponding whether the pixels are the part of main subject (sky) 
(col. 10 lines 32-67). 

"As to claim 105, Luo further discloses segmenting the 
image into plurality of regions and assigning a believe value to the pixels 
corresponding to the level of saliency (col. 9 lines 15-42). 

"As to claims 94-97, Luo further discloses altering color 
saturation (col. 16, lines 2-14), hue (color, col. 10, lines 32-67, note that 
adding blue to pixels in the void or eliminating color pixels not in sky (set 
to respective luminance) can be viewed as altering hue), luminance and 
pixel blur (abstract, column 10, lines 48-67, column 15, lines 26-37). 

"As to claims 106-120, the claims are corresponding system 
claims to claims 91-105. The discussions are addressed with regard to 
claims 91-105. 

"As to claim 121, Luo discloses a method of modifying an 
image having pixels, comprising: 

"automatically; generating one or more belief values, each 
being associated with one of plurality of regions of the image, said belief 
values each being related to the probability that the associated region is a 
main subject of the image (figure 2, 201 , figure 4a-d, column 8, lines 47- 
64); 

"following said generating, automatically altering pixel 
values in the plurality regions of the image, in accordance with the 
associated belief values (fig. 6^ 75, col. 10, lines 49-67). 

"As to claim 1 26, the limitations are addressed with regard 

to claim 121. 

"As to claims 122-125 and 127-130, the limitations are 
addressed with regard to claims 94-105." 



Amended Claim 91 states: 

91 . A method for modifying an image having pixels, 
comprising the steps of: 

automatically identifying a main subject of the image, and 
automatically altering pixel values of said image to 
emphasize said main subject, said altering following said identifying; 

wherein said steps are without manual intervention and said 
altering follows any and all identifying of said main subject. 
Claim 91 is supported by the application as filed, notably the original claims and at 
page 14, line 28 to page 15, line 3. 

Claim 91 requires that the altering follows any and all identifying of 
the main subject. This feature is distinct from Luo (and from the claims that were 
appealed), since the multi-stage identification process of Luo is precluded. (See 
Luo, col. 5, lines 2-3; col. 10, lines 5 and 49; and the discussion in the Decision on 
Appeal, paragraph beginning on page 6 and concluding on page 7) 
Claims 92 and 1 00 were cancelled. 

Claims 93-99 and 101-105 are allowable as depending fi-om Claim 
91 and as follows. 

Claims 94-97 state: 

94. The method of Claim 91 wherein said altering pixel 
values further comprises altering pixel color saturation. 

95. The method of Claim 91 wherein said altering pixel 
values further comprises altering pixel hue. 

96. The method of Claim 91 wherein said altering pixel 
. values further comprises altering pixel luminescence. 

97. The method of Claim 91 wherein said altering pixel 
values further comprises altering pixel blur. 

Unlike Claims 94-97, Luo does not disclose altering saturation, luminance, hue, 
and blur in an altering step that follows any and all identifying of a main subject. 
Luo instead uses features, such as color, brightness, and texture, in the 
identification of regions. (See Luo, Fig. 2; col. 7, lines 9-10; col. 7, lines 46-47; 
col. 8, lines 2-8; col. 6, lines 51-57; col. 8, line 40-col. 9, line 31.) In identifying 
features, Luo produces a list of segmented regions ranked in belief order. The list 
can be converted into a map in which brightness of the regions shows belief order. 

-12- 



(Luo, col. 15, lines 29-3 1 ; col. 3, line 66 to col. 4, line 4. The map can be output. 
Luo, col. 15, lines 40-44; col. 17, lines 1-3. See col. 15-17 generally.) The map 
remains part of the identifying, even if identifjdng is repeated. 

Claims 101-104 all relate to the altering of pixel values of one of 
the properties discussed in relation to Claims 94-97 and are allowable on the same 
grounds as those claims. 

Claim 106 is supported and allowable on the same grounds as 
Claim 91. Claims 107-120 were cancelled. 
Claim 121 states: 

121. A method of modifying an image having pixels, 
comprising the steps of: 

automatically generating one or more belief values, each 
said belief value being associated with one of a plurality of regions of the 
image, said belief values each being related to the probability that the 
associated region is a main subject of the image; 

following said generating, automatically altering pixel 
values in said plurality of regions of said image, in accordance with said 
associated behef values; 

wherein said steps are without manual intervention and said 
altering follows all said generating. 
Claim 121 is supported in the same manner as Claim 91 . 

Claim 121 requires generating belief values that relate to the 
probability that an associated region is a main subject and that altering pixels 
follows "all said generating" and is allowable on the same grounds as Claim 91. 

Claims 122-125 are allowable as depending from Claim 121 and on 
grounds discussed above in relation to Claims 94-97. 

Claims 126-130 were cancelled. 

Added Claims 153-156 are allowable as depending from Claim 91 

and as follows. 

Claim 153 states: 

153. The method of Claim 91 wherein said main subject 
includes skin tone pixels. 
Claim 153 is supported by the application as filed, notably the original claims and 
at page 12, line 9 to page 13, line 3. Claim 153 requires that the main subject 
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defined by the computed main subject belief map includes skin tone pixels. Luo 
excludes non-sky pixels, even if similar in color. (See, for example, Luo, col. 15, 
lines 49-52) 

Claims 154-155 states: 

154. The method of claim 93 wherein said segmenting 
further comprises extracting features of said input digital image and said 
generating further comprises using a reasoning engine to compute said 
belief map from said features. 

155. The method of claim 154 wherein said reasoning 
engine is Bayes net-based. 

Claims 154-1 55 are supported by the application as filed, notably the original 
claims and at page 8, lines 7-8 and page 9, lines 28-29. 
Claim 156 states: 

156 (new). The method of claim 154 wherein said features 
include semantic features and structural features, said semantic features 
including skin, said structural features including centrality. 
Claim 156 is supported by the application as filed, notably the original claims and 
at page 9, lines 21-23; page 12, line 9 to page 13, line 3; and page 10, line 3 to 
page 11, line 3. Claim 156 requires the extracted semantic feature of skin and the 
extracted structural feature of centrality. Luo excludes non-sky pixels even if 
similar in color (See, for example, Luo, col. 15, lines 49-52) and looks to traces 
extending from horizon-to-zenith. (Luo, col. 11, line 1 to col. 12, line 15) 
Claim 157 states: 

157. A digital image modification method for use on an 
input digital image to produce an output image, the method comprising the 
steps of: 

receiving a user selection of one of a plurality of different 
types of emphasizing to provide a selected emphasis; 

identifying a main subject and a background in the input 

digital image; 

altering pixel values of said input digital image, in 
accordance with said selected emphasis, to provide the output image 
wherein said main subject is emphasized; 



wherein said identifying and altering steps are without 
manual intervention. 

Claim 157 is supported by the application as filed, notably the original claims and 
at page 21, lines 15-22 and page 21, lines 27-28. 

Claim 157 requires receiving a user selection of one of a plurality 
of different types of emphasizing to provide a selected emphasis. Pixels are then 
altered in accordance with the selected emphasis. Luo lacks any such step, nor is it 
apparent how such a receiving step would be compatible with steps utilizing a 
physical model of the sky, 

Claims 158-163 are allowable as depending firom Claim 157 and as 

follows. 

Claims 158-159 state: . 

158. The method of claim 157 further comprising 
delivering said output image to an output device. 

159. The method of claim 1 58 wherein said output device is 
a printer or a display. 

Claims 158-1 59 is supported by the application as filed, notably at page 21, line 28 
to page 22, line 2. Claim 158 requires delivering an output image having pixel 
values altered "wherein said main subject is emphasized" to an output device. 
Claim 1 59 requires that the output device is a printer or a display. Luo does 
disclose these features. Figure 1 of Luo shows modification of an input image, but 
that modification is part of a larger system that uses the results of the sky detection 
process. Luo states: . 

"The subject matter of the present invention relates to digital 
image understanding technologv . which is understood to mean technology 
that digitally processes a digital image to recognize and thereby assign 
useful meaning to human understandable objects, attributes or conditions 
and then to utilize the results obtained in the further processing of the 
digital image. 

"A block diagram of the overall sky detection system (e.g., 
the digital image understanding technology) is shown in FIG. 1. First, a 
digital image 10 is digitally processed 20. The results 30 obtained fi*om 
processing step 20 are used along with the original digital image 1 0 in an 
image modification step 40 to produce a modified image 50. 
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"A more specific block diagram of the inventive sky 
detection process is shown in FIG. 2." (Luo, col. 4, line 55 to col. 5, line 2) 
Luo is silent as to the nature of the modification in step 40. 

Claim 160 is supported and allowable on the same grounds as 

Claim 153. 

Claims 164-165 state: 

164. A digital image modification method for use on an 
input digital image in a computer system to produce an output image, the 
method comprising the steps pf: 

segmenting the input digital image into a plurality of 
regions, said regions each being homogeneous; 

computing a main subject belief map of said regions, said 
map defining a main subject in said input digital image; 

then, altering pixel values in said input digital image, in 
accordance with said map, to provide the output image wherein said main 
subject is emphasized; and 

next, delivering said output image to an output device; 

wherein said steps are without manual intervention. 

165. The method of claim 164 wherein said output device is 
a printer or a display. 

Claims 164-165 are supported and allowable on the grounds discussed in relation 
to Claims 158-159. Claim 164 is also allowable as requiring an automated step of 
outputting the output image to an output device. This feature is supported by the 
application as filed, notably at page 22, lines 1-2. This feature is not disclosed by 
Luo. 

Claims 165-172 are allowable as depending fi-om Claim 164 and as 
above and below. 

Claim 166 states: 

166. The method of claim 164 wherein said computing of 
said map is based upon degree^ of centrality, bordemess, and chrominance 
space distribution. 

Claim 166 is supported by the application as filed, notably page 10, line 3 to page 
13, line 3. Claim 166 requires that the main subject defined by the computed main 
subject belief map is based upon degree of centrality, bordemess, and chrominance 

. -16- 



r 

space distribution. This contrasts with the approach of Luo, which looks to traces 
extending from horizon-to-zenith. (Luo, col. 11, line 1 to col. 12, line 15) 

Claim 168 is supported and allowable on the same grounds as 

Claim 153. 

Claim 170 is supported and allowable on the same grounds as 

Claim 156. 

Claims 171-172 state: 

171. The method of claim 1 70 wherein said main subject 
has a high degree of said centrality. 
Claim 171 is supported by the application as filed, notably at page 10, line 3 to 
page 11, hne 3. Claim 171 requires that the main subject has a high degree of 
centrality. This contrasts with Luo, in which the sky is detected using traces 
extending from horizon-to-zenith. (Luo, col. 11, line 1 to col. 12, line 15) 

Claim 1 72 is supported and allowable on the same grounds as 

Claim 153. 

It is believed that these changes now make the claims clear and 
definite and, if there are any problems with these changes. Applicants' attomey 
would appreciate a telephone call. 

In view of the foregoing, it is believed none of the references, taken 
singly or in combination, disclose the claimed invention. Accordingly, this 
application is believed to be in condition for allowance, the notice of which is 
respectfiiUy requested. 



RespectfiiUy submitted. 




Attomey for Applicant(s) 
Registration No. 30,700 
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-17- 



